Comp236: Computational Learning Theory the Winnow Algorithm
نویسنده
چکیده
These notes are slightly edited from scribe notes in previous years. In lecture we only covered sections 1 and 3 of these notes, but we provide the complete details for completeness. 1 The Winnow Algorithm Like the Perceptron algorithm, The Winnow Algorithm learns linear threshold hypotheses. The algorithm and its analysis are specialized to inputs in {0, 1} n , that is, when the features are binary. We think of w as the hypothesis while 1 < α ∈ R and 0 < θ ∈ R are parameters of the algorithm.
منابع مشابه
On Online Learning of Decision Lists
A fundamental open problem in computational learning theory is whether there is an attribute efficient learning algorithm for the concept class of decision lists (Rivest, 1987; Blum, 1996). We consider a weaker problem, where the concept class is restricted to decision lists with D alternations. For this class, we present a novel online algorithm that achieves a mistake bound of O(r log n), whe...
متن کاملText Chunking based on a Generalization of Winnow
This paper describes a text chunking system based on a generalization of the Winnow algorithm. We propose a general statistical model for text chunking which we then convert into a classification problem. We argue that the Winnow family of algorithms is particularly suitable for solving classification problems arising from NLP applications, due to their robustness to irrelevant features. Howeve...
متن کاملLearning Disjunctions of Features
Littlestone's Winnow algorithm for learning disjunctions of Boolean attributes where most attributes are irrelevant is one of the most fundamental results in learning theory. Because of the importance of Winnow, it is natural to try extending Winnow to learn disjunctions of features (i.e., concepts). In doing so, we may generalize some earlier applications of Winnow. In this paper, we extend Wi...
متن کاملComputational Learning Models For Drifting Concepts
In this paper, we explore computational learning models for learning concepts classes where the target concept changes with time, a scenario known in the literature as concept drift. The importance of this topic is immediately evident in the dynamic nature of real-world concepts; if a phenomenon can be represented by a static concept, then it often does not even require learning. The most obvio...
متن کاملOnline Rule Learning via Weighted Model Counting
Online multiplicative weight-update learning algorithms, such as Winnow, have proven to behave remarkably for learning simple disjunctions with few relevant attributes. The aim of this paper is to extend the Winnow algorithm to more expressive concepts characterized by DNF formulas with few relevant rules. For such problems, the convergence of Winnow is still fast, since the number of mistakes ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2013